The present invention relates to information processing by signal processors connected by processing junctions, and more particularly, to neural network models that simulate and extend biological neural networks.
A biological nervous system comprises a complex network of neurons that receive and process external stimuli to produce, exchange, and store information. A neuron, in its simplest form as a basic unit for a neural network, may be described as a cell body called soma, having one or more dendrites as input terminals for receiving signals and one or more axons as output terminals for exporting signals. The soma of a neuron processes signals received from dendrites to produce at least one action signal for transmission to other neurons via axons. Some neurons have only one axon which repeatedly splits branches, thereby allowing one neuron to communicate with multiple other neurons.
One dendrite (or axon) of a neuron and one axon (or dendrite) of another neuron are connected by a biological structure called a synapse. Hence, a neural network comprises a plurality of neurons that are interconnected by synapses. Signals are exchanged and processed within such a network.
Neurons also make anatomical and functional connections with various kinds of effector cells such as muscle, gland, or sensory cells through another type of biological junctions called neuroeffector junctions. A neuron can emit a certain neurotransmitter in response to an action signal to control a connected effector cell so that the effector cell reacts accordingly in a desired way, e.g., contraction of a muscle tissue.
The structure and operations of a biological neural network are extremely complex. Many physical, biological, and chemical processes are involved. Various simplified neural models have been developed based on certain aspects of biological nervous systems. See, Bose and Liang, xe2x80x9cNeural network fundamentals with graphs, algorithms, and applications,xe2x80x9d McGraw-Hill (1996). A brain, for example, is a complex system and can be modeled as a neural network that processes information by the spatial and temporal pattern of neuronal activation.
One description of the operation of a general neural network is as follows. An action potential originated by a presynaptic neuron generates synaptic potentials in a postsynaptic neuron. The soma membrane of the postsynaptic neuron integrates these synaptic potentials to produce a summed potential. The soma of the postsynaptic neuron generates another action potential if the summed potential exceeds a threshold potential. This action potential then propagates through one or more axons as presynaptic potentials for other neurons that are connected. The above process forms the basis for information processing, storage, and exchange in many neural network models.
Action potentials and synaptic potentials can form certain temporal patterns or sequences as trains of spikes. The temporal intervals between potential spikes carry a significant part of the information in a neural network.
Another significant part of the information in a neural network is the spatial patterns of neuronal activation. This is determined by the spatial distribution of the neuronal activation in the network. It is desirable to stimulate both the temporal and spatial patterns in a neural network model. See, for example, Deadwyler et al., xe2x80x9cHippocampal ensemble activity during spatial delayed-nonmatch-to-sample performance in rats,xe2x80x9d Journal of Neuroscience, Vol. 16, pp.354-372 (1996) and Thiels et al., xe2x80x9cExcitatory stimulation during postsynaptic inhibition induces long-term depression in hippocampus in-vivo,xe2x80x9d Journal of Neuroscience, Vol. 72, pp.3009-3016 (1994) and xe2x80x9cNMDA receptor-dependent LTD in different subfields of hippocampus in vivo and in vitro,xe2x80x9d Hippocampus, Vol. 6, pp. 43-51 (1996).
Many neural network models are based on the following two assumptions. First, synaptic strength, i.e., the efficacy of a synapse in generating a synaptic potential, is assumed to be static during a typical time scale for generating an action potential in neurons. The efficacy of a synapse is essentially a constant during a signal train. Certain models modify this assumption by allowing a slow variation over a period of processing many signal trains. In the second assumption, each sending neuron provides the same signal to all other neurons to which it is synaptically connected.
One aspect of the present invention provides an improved neural network model that removes the above two assumptions and enables neural network devices to perform complex tasks. The present invention includes information processing systems and methods that are inspired by and are configured to extend certain aspects of a biological neural network. The functions of signal processors and processing junctions connecting the signal processors correspond to biological neurons and synapses, respectively. Each of the signal processors and processing junctions may comprise any one or a combination of an optical element, an electronic device, a biological unit, or a chemical material. The processing systems and methods may also be simulated by using one or more computer programs.
Each processing junction is configured to dynamically adjust its response strength according to the temporal pattern of an incoming signal train of spikes. Hence, such a processing junction changes its response to the incoming signal and hence simulates a xe2x80x9cdynamic synapsexe2x80x9d.
Different processing junctions in general respond differently to the same input signal. This produces different output junction signals. This provides a specific way of transforming a temporal pattern of a signal train of spikes into a spatio-temporal pattern of junction events. In addition, the network of the signal processors and processing junctions can be trained to learn certain characteristics embedded in input signals.
One embodiment of a system for information processing includes a plurality of signal processors connected to communicate with one another and configured to produce at least one output signal in response to at least one input signal, and a plurality of processing junctions disposed to interconnect the signal processors. Each of the processing junctions receives and processes a prejunction signal from a first signal processor in the network based on at least one internal junction process to produce a junction signal which causes a postjunction signal to a second signal processor in the network. Each processing junction is configured so that the junction signal has a dynamic dependence on the prejunction signal.
At least one of the processing junctions may have another internal junction process that makes a different contribution to the junction signal than the internal junction process.
Each of the processing junctions may be connected to receive an output signal from the second signal processor and configured to adjust the internal junction process according to the output signal.
These and other aspects and advantages of the present invention will become more apparent in light of the following detailed description, the accompanying drawings, and the appended claims.